This paper presents a decentralized leader-follower multi-robot formation control based on a reinforcement learning (RL) algorithm applied to a swarm of small educational Sphero robots. Since the basic Q-learning method is known to require large memory resources for Q-tables, this work implements the Double Deep Q-Network (DDQN) algorithm, which has achieved excellent results in many robotic problems. To enhance the system behavior, we trained two different DDQN models, one for reaching the formation and the other for maintaining it. The models use a discrete set of robot motions (actions) to adapt the continuous nonlinear system to the discrete nature of RL. The presented approach has been tested in simulation and real experiments which show that the multi-robot system can achieve and maintain a stable formation without the need for complex mathematical models and nonlinear control laws.

Sphero's been amusing us with its collection of robotic balls, like its adorable BB-8, for eight years. But lately the company has been getting away from the toy aspect of its products and embracing its educational potential. It's had an app that can be used to program many of its current bots for a while now, but that's only for budding coders -- what do kids interested in hardware have to tinker with? Indeed, Sphero is about to release its first robot specifically made to be physically modded, called the RVR. RVR -- that's pronounced "Rover" -- is a big change for Sphero.

You probably know the Sphero robot. It is a small robot with the shape of a ball. In case that you have one, you must know that it is possible to control it using ROS, by installing in your computer the Sphero ROS packages developed by Melonee Wise and connecting to the robot using the bluetooth of the computer. Now, you can use the ROS Development Studio to create ROS control programs for that robot, testing as you go by using the integrated simulation. The ROS Development Studio (RDS) provides off-the-shelf a simulation of Sphero with a maze environment.